Distributional Clustering of Words for Text Classi cation

This paper describes the application of Dis tributional Clustering to document classi cation This approach clusters words into groups based on the distribution of class labels associated with each word Thus unlike some other unsupervised dimensionality reduction techniques such as Latent Semantic Indexing we are able to compress the feature space much more aggressively while still maintaining high document clas si cation accuracy Experimental results obtained on three real world data sets show that we can reduce the feature dimen sionality by three orders of magnitude and lose only accuracy signi cantly better than Latent Semantic In dexing class based clustering feature selection by mutual information or Markov blanket based fea ture selection We also show that less aggressive clustering sometimes results in improved classi cation accuracy over classi cation without clustering